Skip to content

DEV-1337: don't write the clock to a streaming Shimmer3R - #339

Merged
JongChern merged 2 commits into
masterfrom
DEV-1337_no_rtc_write_while_streaming_3r
Oct 29, 2026
Merged

JongChern merged 2 commits into
masterfrom
DEV-1337_no_rtc_write_while_streaming_3r

Conversation

@JongChern

@JongChern JongChern commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

DEV-1337. The driver no longer writes the clock (SET_RWC) to a Shimmer3R that is streaming. That removes the trigger for DEV-1245 from the Java and Android APIs: a Shimmer3R firmware issue, still in v1.1.22, where streamed timestamps go out of order and are sometimes shifted 512 s.

The problem

On a Shimmer3R the counter that stamps each streamed packet is the real-world clock. Writing the clock while the unit streams steps that counter:

  • packets come out off the sample grid, and sometimes behind the one before;
  • a step back beyond the timestamp unwrap's 8-period reorder window (31 ms at 256 Hz) reads as a counter wrap, so every later sample in the session is stamped 512 s late.

startSDLogging() wrote the clock before every START_LOGGING_ONLY_COMMAND. So any Java or Android app that started SD logging while a Shimmer3R streamed triggered it every time. The Android HIL's midStreamSet and inStream do that, and they were the only failures in the 24 Oct Android release gate.

The change

  • The check: writeRealTimeClock() queues nothing while a Shimmer3R streams (isRealTimeClockWriteBlocked()), and logs that it didn't. Every clock write goes through it: startSDLogging(), an app's own call, and the status rule. The IO thread is unchanged.
  • What a queue-time check misses: a clock write queued just before streaming starts, which goes out once it has. That needs an app to call startSDLogging() between startStreaming() and its ACK.
  • The clock is still set: the connect sequence writes it, as before. If a status reply ever shows it unset, the status rule writes it at the next reply while the unit isn't streaming. Setting it at connect rather than at the logging start costs no meaningful accuracy: the Bluetooth write itself is only good to tens of milliseconds.
  • Shimmer3 is unchanged: its timestamps stay in order across a clock write, so it still gets the write mid-stream.
  • Docs: the javadoc for writeRealTimeClock() and startSDLogging(), and the Java HIL README's DEV-1245 note, are updated.

Other APIs

None of the others writes the clock mid-stream, so this brings Java and Android into line with them:

  • the C# API writes it only at connect (ShimmerLogAndStream, when not logging);
  • pyshimmer's start_logging() and the web SDK's logging start don't write it.

Android's Classic Shimmer (with its own copy of the status rule) and Shimmer3BLEAndroid both write the clock through writeRealTimeClock(), so Android picks this up with the next driver release and a pin bump. At that bump, AAPI_00002, AAPI_00003 and AAPI_00005 need the same change as the three tests below, because they expect startSDLogging() to queue SET_RWC on a streaming Shimmer3R.

Tested

Unit test: API_00035_NoClockWriteWhileStreamingTest runs the real IO thread over an in-memory radio and records what reaches the unit.

Case master this branch
startSDLogging() on a streaming Shimmer3R fails: SET_RWC sent passes: only START_LOGGING_ONLY
writeRealTimeClock() on a streaming Shimmer3R fails: SET_RWC sent passes: nothing sent
startSDLogging() on an idle Shimmer3R passes: SET_RWC, then START_LOGGING_ONLY passes, unchanged
startSDLogging() on a streaming Shimmer3 passes: SET_RWC sent first passes, unchanged
connect to a unit that isn't logging passes: SET_RWC queued passes, unchanged
connect to a logging unit passes: no SET_RWC passes, unchanged

Three existing tests changed: API_00027, API_00030 and API_00032 put a SET command in flight by calling startSDLogging() on a streaming Shimmer3R. That used to queue SET_RWC first. They now use the START_LOGGING_ONLY it queues alone. Where a test checks that the next command survives, GET_VBATT is queued behind it.

ShimmerDriverPC (103 tests) and ShimmerDriver pass.

Hardware: s3r-2f31 (Shimmer3R, LogAndStream v1.1.22) over BLE, from an M2101K6G phone on Android 11, through the Android HIL. This branch's driver was published locally as 0.11.12-dev1337; master's is 0.11.11_beta. Each scenario ran twice per driver.

These runs used this branch's first commit, which dropped the clock write in the IO thread. Both scenarios call startSDLogging() once the unit has streamed for 3 s, so a check where the write is queued blocks the same writes.

Driver midStreamSet inStream SET_RWC per run
0.11.11_beta fails 2 of 2: 511.9 s timestamp jumps, a backward timestamp fails 1 of 2: a backward timestamp. The other run had 2 jumps that inStream only reports, not fails, near an SD start 4–5 sent, 0 dropped
this branch passes 2 of 2: no backward timestamps, no jumps passes 2 of 2: no backward timestamps, no jumps 1 sent (at connect), 3–4 dropped (mid-stream)

🤖 Generated with Claude Code

JongChern and others added 2 commits October 25, 2026 12:46
On a Shimmer3R the counter that stamps each streamed packet is the
real-world clock, so a clock write (SET_RWC) while it streams steps that
counter. Packets come out off the sample grid and sometimes behind the one
before, and a step back beyond the timestamp unwrap's reorder window reads
as a counter wrap, stamping the rest of the session 512 s late (DEV-1245,
firmware, still in v1.1.22). startSDLogging() wrote the clock before every
logging start, so starting logging mid-stream triggered it every time.

The IO thread now drops a clock write unsent while a Shimmer3R streams,
whatever queued it: startSDLogging(), an app's writeRealTimeClock(), or the
status rule. Checking at send time also catches a write queued just before
streaming started. The connect sequence still writes the clock, and the
status rule writes it at the next reply while the unit isn't streaming if
it is ever unset. A Shimmer3 keeps its timestamps in order across a clock
write, so it still gets it mid-stream.

The C# API already writes the clock only at connect, and pyshimmer and the
web SDK don't write it when logging starts.

API_00035 runs the IO thread over an in-memory radio. Without the fix, a
streaming Shimmer3R is sent SET_RWC by startSDLogging() and by
writeRealTimeClock(); with it, neither is, while an idle Shimmer3R, a
streaming Shimmer3 and the connect-time write are unchanged.

On a Shimmer3R over BLE (Android HIL midStreamSet and inStream, two runs
each), driver 0.11.11 failed 3 of 4 with backward timestamps and 512 s
jumps; this build passed 4 of 4, sending the clock once at connect and
dropping the 3-4 mid-stream writes per run.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
Move the check out of the IO thread: writeRealTimeClock() now queues nothing
for a Shimmer3R that is streaming, and processNextInstruction() is back as it
is on master. Every clock write goes through writeRealTimeClock(): the status
rule, startSDLogging(), and Android's Shimmer.java.

API_00027, API_00030 and API_00032 put a SET command in flight with
startSDLogging() on a streaming Shimmer3R, which queued SET_RWC_COMMAND first.
They now use the START_LOGGING_ONLY_COMMAND it queues alone, with
GET_VBATT_COMMAND queued behind it where a test checks the next command
survives.

Co-Authored-By: Claude Opus 5.5 <[email protected]>
@JongChern
JongChern merged commit 5148a2a into master Oct 29, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant